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ABSTRACT 

Context. One of the main challenges of modern cosmology is to understand the nature of the mysterious dark energy 
which causes the cosmic acceleration. The Integrated Sachs- Wolfe (ISW) effect is sensitive to dark energy and if detected 
in a universe where modified gravity and curvature are excluded, presents an independent signature of dark energy. The 
ISW effect occurs on large scales, where cosmic variance is high and where there are large amounts of missing data in 
the CMB and large scale structure maps due to Galactic confusion. Moreover, existing methods in the literature often 
make strong assumptions about the statistics of the underlying fields or estimators. Together these effects can severely 
limit signal extraction. 

Aims. We want to define an optimal statistical method for detecting the ISW effect, which can handle large areas of 
missing data and minimise the number of underlying assumptions made about the data and estimators. 
Methods. We first review current detections (and non-detections) of the ISW effect, comparing statistical subtleties 
between existing methods, and identifying several limitations. We propose a novel method to detect and measure the 
ISW signal. This method assumes only that the primordial CMB field is Gaussian. It is based on a sparse inpainting 
method to reconstruct missing data and uses a bootstrap technique to avoid assumptions about the statistics of the 
estimator. It is a complete method, which uses three complementary statistical methods. 

Results. We apply our method to Euclid-like simulations and show we can expect a ~ 7a model-independent detection 
of the ISW signal with WMAP7-like data, even when considering missing data. Other tests return ~ 4.7<r detection 
levels for a Euclid-like survey. We find detections levels are independent from whether the galaxy field is normally or 
lognormally distributed. We apply our method to the 2 Micron All Sky Survey (2MASS) and WMAP7 CMB data and 
find detections in the 1.1 — 2. Oct range, as expected from our simulations. As a by-product, we have also reconstructed 
the full-sky temperature ISW field due to 2MASS data. 

Conclusions. We have presented a novel technique, based on sparse inpainting and bootstrapping, which accurately 
detects and reconstructs the ISW effect. 

Key words. ISW, inpainting, signal detection, hypothesis test, bootstrap 



1. Introduction 

The recent abundance of cosmological data in the last few 
decades (for an example of the most recen t results see 
Komatsu et~aT1 [20091 |Percival et aLl|2007 a[ |Schrabback| 



et al. 2010) has provided compelling evidence towards a 
standard concordance cosmology, in which the Universe is 
composed of approximately 4% baryons, 26% 'dark' matter 
and 70% 'dark' energy. 

One of the main challenges of modern cosmology is to 
understand the nature of the mysterious da rk energy which 



drives the observed cosm ic acceleration ( |Albrecht et al 
[2006] [Peacock et al.|2006[ 



The Integrated Sachs-Wolfe (ISW) ( |Sachs fc Wolfe 
19671 effect is a secondary anisotropy of the Cosmic 
Microwave Background (CMB), which arises because of the 
variation with time of the cosmic gravitational potential 
between local observers and the surface of last scattering. 
The pot ential can be traced by Larg e Scale Structure (LSS) 
surveys (Crittenden & Turok 1996), and the ISW effect is 
therefore a probe which links the high redshift CMB with 
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the low redshift matter distribution and can be detected by 
cross-correlating the two. 

As a cosmological probe, the ISW effect has less statis- 
tical power than weak lensi ng or galaxy clustering (see for 



Refregier et al. 20101, but it is direc tly sensitive to 



dark energy 


curvatm 


-e or modified gravity ( 


Kamionkowski 


& Spergel 


1994 Kinkhabwala & Kamionkowski 1999 


Carroll et al. 


2005 


Song et al. 


2007 


), such that in uni- 



detection of the ISW signal provides a direct signature of 
dark energy. In more general universes, the ISW effect can 
be used to trace alternative models of gravity. 

The CMB WMAP survey is already optimal for detect- 
ing the ISW signal (see Sections [2] and [3]) , and significance 
is not expected to increase with the arrival of Planck, un- 
less the effect of the foreground Galactic mask can be re- 
duced. The amplitude of the measured ISW signal should 
however depend strongly on the details of the local tracer 



of mass. Survey optimisations (Douspis et al. 20081 show 
that an ideal ISW survey requires the same configuration 
as surveys which are optimised for weak lensing or galaxy 
clustering - meaning that an optimal measure of the ISW 
signal will essentially come 'for free' with future planned 
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weak lensing a nd galaxy clustering s urveys (see for e.g. the 
Euclid survey Refregier et aL 2010D. In the best scenario, 
a 4ct detection is expected QDouspis et ah] 2008 ) , and it 
has been shown that combined with weak lensing, galaxy 
correlation and other probes such as clusters, the ISW can 



be us eful to break parameter degeneracies (|Refregier et al. 
2010b, making it a promising probe 



so that such a blue- and red-shift will always cancel out 
because <&' = and there will be no net effect on the wave- 
length of the photon. 

H owever, in the presence of dark energy or curva- 
ture dSachs fc Wolfe|[l967l |Kamionkowski k, Spergel||1994 



Kinkhabwala & Kamionkowski 1999[ ), the right hand side 
of Equation |l | will be non-null a s the cosmic po tential will 



Initial attempts to detect the ISW effect with COBE change with time (see for e.g., Dodelson||2003|) , resulting 



as th e CMB tracer were fruitless (Boughn &; Crittenden 
2002 1 , but since the arrival of WMAP data, tens of positive 



detections have been made, with the highest significance 
reported for analyses using a tomographic combination of 
surveys (see Sections [2] and [3] for a detailed review of de- 
tections) . However, several studies using the same tracer of 
LSS appear to have contradicting conclusions, some analy- 
ses do not find correlation where others do, and as statisti- 
cal methods to analyse the data evolve, the signi ficance of 
the ISW signal is sometimes reduced (see for e.g ., |Afshordi| 



et al.||2004[ |Rassat et aT|2007| |Francis fc Peacock||2010b 



rdi 
)• 



In Section |2J we describe the cause of the ISW effect 
and review current detections. In Section[3] we describe the 
methodology for detection and measuring the ISW signal, 
and review a large proportion of reported detections in the 
literature, as well as their advantages and disadvantages. 
Having identified the main issues with current methods, we 
propose a new and complete method in Section [4j which 
capitalises on the fact that different statistical methods are 
complementary and uses sparse inpainting to solve the issue 
of missing data and a bootstrapping technique to measure 
the estimator's probability distribution function (PDF). In 
Section [5j we validate our new method using simulations 
for 2MASS and Euclid-like surveys. In Section |6j we apply 
our new method to WMAP 7 and the 2MASS survey. In 
Section [7j we present our conclusions. 



2. The Integrated Sachs-Wolfe Effect 

2.1, Origin of the Integrated Sachs-Wolfe Effect 

General relativity predicts that the wavelength of electro- 
magnetic radiation is sensitive to gravitational potentials, 
an effect which is called gravitational redshift. Photons trav- 
elling from the surface of last scattering will necessarily 
travel through the gravitational potential of Large Scale 
Structure (LSS) on their way to the observer; these will 
be blue-shifted as they enter the potential well and red- 
shifted as they exit the potential. These shifts will accumu- 
late along the line of sight of the observer. The total shift 
in wavelength will translate into a change in the measured 
temperature-temperature anisotropy of the CMB, and can 
be calculated by: 



(") 

V 1 J ISW 



■no 

2 / $' ((rjo - r))n, rj) dr), 

1 t}L 



(1) 



where T is the temperature of the CMB, r\ is the conformal 
time, defined by d-q — and ry an d Vl represent the 
conformal times today and at the surface of last scattering 
respectively. The unit vector n is along the line of sight 
and the gravitational potential ^(x, rf) depends on position 
and time. The integral depends on the rate of change of the 
potential $' = d$/dry. 

In a universe with no dark energy or curvature, the cos- 
mic (linear) gravitational potential does not vary with time, 



in a secondary anisotropy in the CMB temperature field 



2.2. Detection of the ISW signal 

The ISW effect leads to a linear scale secondary anisotropy 
in the temperature field of the CMB, and will thus af- 
fect the CMB temperature power spectrum at large scales. 
Due to the primordial anisotropics and cosmic variance on 
large scales, the ISW signal is difficult to detect directly 
in the tempe rature map of the CMB, but Crittenden fc| 



Turok ( 1996 ) showed it could be detected through cross- 
correlation of the CMB with a local tracer of mass. 



The first a t tempt to detect the ISW effect (Boughn 
& Crittenden 2002 ) involved correlating the Cosmic 



Microwave Backgro und explorer data ( Bennett et al.|[ 1990 
COBE) wit h XRB ( [Bddlfl987) and WHH data ( [(Jonclon 



et al.||1998[ ). This analysis did not find a significant cor 
relation between the local tracers of mass and the CMB. 
Since the release o f data from the Wil kinson Microwave 
Anisotropy Probe flSpergel et~aT1|2003| WMAP) over 20 
studies (see Table [T]) have investigated cross-correlations 
between the different years of WMAP data and local 



trace rs selected using vario us wavelengths: X-ray ( Boldt 
1987, XRB survey); opt ical ( Agiieros et al.|2006 Adelman- 
|McCa rthy et al.j|2008[ SPSS galaxies), QAnderson et aT 



20011 SPSS QSWTlDoroshkevich et al.| |2004l SPSS 



ct al 



ERTJs), (Maddox et al.||1990[ APM); near infrar ed ( |Jarrett 
-| |2000l 2MAijjj); radio ( [Condon et al.||1998l NVSS). 
he full sky WMAP data have sufficient resolution < 



The full sky 

large scales that the measure of the ISW signal is cos- 
mic variance limited. The best LSS probe of the ISW ef- 
fect should include maximum sky coverage and full redshift 



cover age of the dark energy dominated era (Douspis et al. 
|2008[ ). No such survey exists yet, so there is room for im- 
provement on the ISW detection as larger and larger LSS 
surveys arise. For this reason, when we review the current 
ISW detections, we classify them according to their tracer 
of LSS, and not the CMB map used. 

The measure of the ISW signal can be done in vari- 
ous statistical spaces; we classify detections in Table [T] into 
three measurement 'domains': Dl corresponds to spheri- 
cal harmonic space; D2 to configuration space and D3 to 
wavelet space. (In Section [3| we review the different meth- 
ods for quantifying the statistical significance of each mea- 
surement) . 

There are only two analyses which use COBE as CMB 



data (with XRB and NVSS data, Boughn fc Crittenden 



2002), and both report null detections, which can reason 



ably be due to the low angular resolution of COBE even 
at large scales. The rest are done correlating WMAP data 
from years 1, 3 and 5 (respectively 'Wl', 'W3' and 'W5' in 
table 0). 

Most ISW detections reported in Table [T] are relatively 
'weak' (< 3cr) and this is expected from theory for a con- 
cordance cosm ology. Higher detections are reported for the 
NVSS survey (iPietrobon et al.||2006| iMcEwen et al. 112008 



F.-X. Dupe et al.: Measuring the Integrated Sachs Wolfe Effect 



3 



Author 




CMB 


LSS Tracer 


Wavelength 


Method 


Claimed 
Detection 


1 5 1 

Boughn & Crittenden 


( zUUz ) 




YD TD 


Xray 


JJZ 


l\ o 


Uiannantomo et al.| (|2 


doslr 


W3 






D2 


2.7a 


Boughn & Crittenden 


(2004, 2005) 


Wl 


XRB/NVSS 


Xray/Radio 


D2 


'tentative' (2-3 a) 


Fosalba et al. ( 2003|) 




Wl 


SDSS DR1 




D2 


2a (low z) 














3.6a (high z) 



Cabre et al. I ll ' 
Uiann antomo et al.l (12008 1 
Sawangwit et al.|{20lU| 
Ldpez-Uorredoifa et al.|fl2010| 



006| 

et al. 



Ciannantonio ct al. 12006) 
Uiann antonio ct al. (2008) 

Xia et al.| (|20U9|> 

Scranton ct~aT~(2003j 



W3 
W3 
W5 
W5 



SDSS DR4 
SDSS DR6 
SDSS DR5 
SDSS DR7 



Optical 



D2 
D2 
D2 
D2 



> 2a 
2.2a 
'marginal' 
'No detection' 



ir'adma nabhan e t al. ( 2005 \ 
Cranett et al.| (|20U9p 
Giannantonio eT al.l ( 2008 1 



Sawangwit et al.| ( [2010| 
Sawangwit et al.l J2010|) 



Fosalba & Caztaiiaga (2004) 
Afshordi ct al. 1 20047 
Rassat et aL] (|20UV[) 
Giann antonio ct al. (2008) 
Francis & Fcacock (2010t 
Boughn & Crittenden ( 2C 
JNolta et aL|(|20U4[) " 
Fietrobon et al.| (|2006[ ) 
Vielva et aL] (|20U5|~ 
McEwen eral.| (|2007[ ) 
Kaccanelli et a l .l ( I 201J 
McEwen ct al. (2008) 
Gianna ntonio et al. (2008) 
Herna ndcz-Moiitcagudo|l2009|) 
S awangwi t et al. | ||2010 |) 
Corasaniti ct al. 1 2005) 
Gaztanaga et a l.l I 
Ho et al.l f2008|) 



(2008) 



W3 
W3 
W5 



SDSS Quasars 
SDSS Quasars 
SDSS Quasars 



Optical 



Wl 
Wl 
W3 
W3 

W5 
W5 



SDSS LRG 



SDSS LRG, 2SLAQ 
AAOmega LRG 



Optical 



Wl 



APM 



Optical 



Wl 
W3 
W3 
W3 



2MASS 



NIR 



COBE 

Wl 

W3 

W3 

W3 

W3 

W3 

W3 

W3 

W5 



NVSS 



Radio 



Wl 
Wl 
W3 
W3 



D2 
D2 
D2 



D2 
Dl 
Dl 
D2 

D2 
D2 



D2 



Dl 
Dl 
D2 
Dl 



D2 
D2 
D3 
D3 
D3 
D2 
D3 
D2 
Dl 
D2 



Combination 



Combination 



D2 
D2 
Dl 
D2 



2a 
2.5a 
2.7a 



> 2a 
2.5a 
2a 
2.2a 

'marginal' 
Null 



2.5a 



2.5 a 
2a 
0.5a 

'weak' 



No 
2.2t7 

> 4(7 

3.3a 
> 2.5a 
2.7a 

~ 4(7 

3.3(7 

< 2(7 

'marginal' (~ 



2a) 



> 2(7 

2a 
3.7a 
4.5cr 



Table 1. Meta-analysis of ISW detections to date and their reported statistical significance. The 'Method' describes 
the space in which the power spectrum analysis is done (configuration, spherical harmonic, etc . . . ), not the method for 
measuring the significance level of the detection (this is described in Section |3|. Dl corresponds to spherical harmonic 
space, D2 to configuration space, D3 to wavelet space. The highest detections are made in wavelet space. Regarding the 
survey used, the highest detections are made using NVSS (though weak and marginal detections using NVSS are also 
reported) or using combinations of LSS surveys as the matter tracer. 



Giannantonio et al. 2008), though weak and marginal de- 
tections using NVSS data are also repo rted (Hernandez- 
Montcagudo 2009; Sawangwit et al.|2010 | High detections 



are often made using a wa velet analysis ( Pietrobon et al. 



2006 McEwen et al. 2008), though a similar study by the 



same authors using the same d ata but a different an alysis 



method finds a weaker signal (McEwen et al. 20071. The 
highest detection is reported using a tomographic combi- 
nation of all surveys (XRB, SDSS galaxies, SDSS QSOs, 



2MASS and NVSS, |Giannantonio et al.|2008| , as expected 
given the larger redshift coverage of the analysis. 



Several analyses have been revisited to seek confirma- 
tion of previous detections. In some cases, results are ver y 
similar (Padmanabhan et al. (20051; Granett et al. (2009); 



Giannantonio et al. (2008), for SDS S LRGs; Giannantonio 
et al.|P006||2008D fo r SPSS Quasars; [Afshordi et al.| ( |2004| y 



Rassat et a 



. p007|) , for 2MASS), but in some casesTEey 
are controversially different ( for e.g. Pietrobon et al. ([2006) 
and |Sawangwit et al.| ([2010D, fo r NVSS or |Afs hordi et al. 
(|2004p andjGiannantoriio et al.| p008|, for 2MASS). 



We also notice that as certain surveys are revisited, 
there is a trend for the statistical significance to be re- 
duced: for e.g., detections from 2MASS decrease from 



2 



a 2.5q det ection ( Afshordi et al. 2004 ) to 2 a (Rassat 
let al.||2007D, to 0.5o~(|Giannantonio et al.||20"08[) to 'weak' 
Francis fe PeacocklpOlO bp. Detections usin g SPSS LRGs 



ecre ase from 2.5cr dPa dmanab han et al. 2005 1, to 2 
2.2cr ( |Granett et al.| 2009[ |Giannantonio et al.| 2008[ ), to 
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'marginal' (Sawangwit et al. 2010). Furthermore, there 
tends to be a 'sociological bias' in the interpretation of 
the confidence on the signal detection. The firs t detec- 
tions interpret a 2 — 3a detection as 'tentative' (Boughn 



& Crittenden 2004 2005), while further studies with sim 
ilar det ection level report 'independent evidence o f dark 
energy' ( |Afshordi et al.|2004| [Gaztanaga et aL||2006| ). 



3. Methodology for Detecting the ISW Effect 

In this paper, we are interested in qualifying the differences 
between different statistical methods which exist in the lit- 
erature, and comparing them with a new method we present 
in Section [4] By statistical method, we mean the method 
which is used to quantify the significance of a signal, not the 
space in which the signal is measured. Therefore, and with- 
out loss of generality, the review presented in Section |3.2 
summarises methods using spherical harmonics. We com- 
pare the pros and cons of each method in Section |3.3| We 
begin by describing how the ISW signal can be measured 
in spherical harmonics in Section 3.1 



3.1. ISW Signal in Spherical Harmonics 

In general, any field can be decomposed by a series of 
functions which form an orthonormal set, as do the spher- 
ical harmonic functions Yg m (0,(f>). Therefore a projected 
galaxy overdensity (S g ) or temperature anisotropy (St) field 
Sx(0,<f>), where X = g,T, can be decomposed into: 



,af m Yi m (e,(j)), 



(2) 



where of m are the spherical harmonic coefficients of the 
field. The 2-point galaxy-temperature cross-correlation 
function can then be written: 



C, 



J — ^Tle [4 m (aJ m )*) , (3) 



(2i + l) 



where taking the real part of the product ensures that 

C gT (£) = C Tg (l). 

The theory for the angular cross-correlation function is 
given by: 

C gT (£) = Anb g J dk^iw g (k)W T (k), (4) 

where 



W T (k) 



W g (k) = J dr9(r)j,(kr)D(z), (5) 



k 2 c 3 



[ i drj,(kr)H(z)D(z)(f-l), (6) 
Jo 

(7) 



A 2 W = 7 ^k 3 P(k), 



(27T) 2 



0(r) 



2 n(r) 



J drr 2 n(r) 



(8) 



In these equations, r represents the co-moving distance, 
Zl the redshift at the last scattering surface, k the Fourier 
mode wavenumber and quantities which depend on the red- 
shift z have an intrinsic dependence on r: H(z) = H(z(r)). 
The function / is the linear growth factor given by / = 



dhi a(zj* ' wnere D(z) is the linear growth which measures 
the growth of structure. The cross-correlation function de- 
pends on the survey selection function given by n(r), in 
units of galaxies per unit volume. The quantities fl m ,o and 
Hq are the values of the matter density and the Hubble 
parameter at z = 0. Units are chosen so that the quantity 
C(£) is unitless. 

In the case where both the temperature and the galaxy 
fields behave as Gaussian random fields, then the covariance 
on the ISW signal can be calculated by: 



a 



gT\ 



1 



[C 2 gT + (C gg +N g )(CTT+M T )}, 

(9) 



where Ctt is the temperature-temperature power spec- 
trum, Af g and Mt are the noise of the galaxy and temper- 
ature fields respectively. The galaxy auto-correlation func- 
tion can be calculated theoretically in linear theory by: 



C gg (£) = 4tt&? 



dk ^!W [ Wg (k)] 2 , (10) 



There are many difficulties in measuring the ISW ef- 
fect, the first being the intrinsic weakness of the signal. To 
add to this, an unknown galaxy bias scales linearly with 
the ISW cross-correlation signal (see Equation [4|, which 
is therefore strongly degenerate with cosmology. Galactic 
foregrounds in both the CMB and the LSS maps also mask 
crucial large scale data and can introduce spurious correla- 
tions. Any method claiming to detect the ISW effect should 
be as thorough as possible in accounting for missing data, 
and where possible the reported detection level should be 
independent of an assumed cosmology. 



3.2. Review on Current Tools for ISW Detection 

In the literature there are two quantities which can be 
used to measure and detect the ISW signal, which we 
review in this section. Without loss of generality, we 
present these methods in spherical harmonic space. The 
first method measures the observ ed cro ss-correlation spec- 
tra ('Spectra' method: see section [3.2.3 ), whilst the second 
directl y comp ares temperature field's 
section 



Fields' method: see 



3.2.4). These two approaches differ by the quantity 



they measure to infer a detection. For each method (fields 
vs. spectra), it is possible to use different statistical meth- 
ods to infer detection which we describe below. We review 
each existing method below and summarise the pros and 
cons of both of these classes as well as the statistical mod- 
els in Tabled 

3.2.1. Note on the confidence score 

Before reviewing the ISW detection methods, we would like 
to clarify the definition of confidence scores from a statisti- 
cal point of view. The confidence of a null hypothesis test 
can be interpreted as the distance from the data to the 
null hypothesis (commonly named Hq). For example, let p 
be a variable of interest (e.g. correlation coefficient, ampli- 
tude). The confidence score a for the hypothesis test H 
(i.e., p = 0) against Hi (i.e., p ^ 0) is directly computed 
using the formula r = p/cr(p), where u(p) is the standard 
deviation. But this is only true when 1) p is Gaussian, 2) 
<j(p) is computed independently from the observation and 
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Measured 
Quantity 


Advantage 


Disadvantage 


Spectra 


Methods exist for calculation with missing data. 
Can introduce tomography easily. 


Assumes C(f)'s or estimator arc Gaussian. 
Most methods requires 
estimation of the covariance matrix. 


Fields 


No assumption about galaxy/matter density field 


Missing data is an ill-posed problem. 


Statistical 

IVlcl 11UU 


Advantage 


Disadvantage 


Simple correlation 
(Spectra / Fields) 


Independent of cosmology. 


Measure of significance 
assumes estimator is Gaussian. 


A m Tili'Hifip pt;i"i "in a i~i r»n 

iilllJJll L U.U.C Co LlllltXLlUll 

(Spectra / Fields) 


Validates signal and model simultaneously. 


l)pi~ppi~~ir\~n Hpr^pnriQ rin rvi q tyi n 1 n tr v /inriripl 

L/C LiCV_. L1U11 UCUC11UB KJLL LAJolllUlU tt, y / 111UU.C1. 

Measure of significance assumes estimator is Gaussian. 
Assumes underlying theory (e.g. ACDiM) is correct. 


X 2 (Spectra / Fields) 


Validates signal and model simultaneously. 


Assumes C(i)'s are Gaussian (Spectra). 
Only gives confidence of rejecting null hypothesis. 
Assumes underlying theory (e.g. ACDM) is correct. 


Model comparison 
(Spectra) 


Asks a different question than other tests. 


ISW signal usually too weak to be detected this way. 
Assumes underlying theory (e.g. ACDM) is correct. 



Table 2. TOP: Review of advantages and disadvantages of measuring spectra vs. fields in order to infer an ISW detection. 
BOTTOM: Review of statistical methods existing in the literature and their respective advantages and disadvantages. 



3) considering a symmetric test. Then, this method does 
not stand for the general case and as the correlation co- 
efficient considered here must be positive, an asymmetric 
(one-sided) test would be more appropriate here. 

As the confidence score is directly linked to the devi- 
ation from the Hq hypothesis through the p-value, the a- 
score is always positive. Then the p-value p of one hypoth- 
esis test is computed using the probability density function 
(PDF) of the test distribution: p = 1 - P(x\H Q )dx = 

f °°P{x\H )dx (for a classical one-sided test). Remark 
that in that case, if the Hq is true then p s» (i.e. in the 
middle of the test distribution) and the p-value p will be 
around 0.5 which correspond to a confidence of 0.67a. 

3.2.2. Note on the application spaces 

All the methods that will be described in the next sections 
can be performed in different domains. While some spaces 
may be more appropriate than others for a specific task, 
difficulties may also arise because of the properties of the 
space. For example: for the 'spectra' method in configura- 
tion space, the two main difficulties are missi ng data and 
the estimation of t he covariance matrix (see e.g. Hernandez- 



Monteagudo| 2008 ) . In this case, the covarianc e matrix can 
be es timated using Monte Carlo methods (see Cabre et al. 
2007). In spherical harmonic space, missing data induce 
mode correlations which can be removed by using an appro- 
priate fram ework for calculation of the C(£)'s (e.g. Hivon 
et al. 2002 1 . In harmonic space (when missing data is ac- 
counted for) the covariance matr ix is diagonal and thus 
easily invertible (see Section 3.2.4). 



3.2.3. Cross-power spectra comparison 

The most popular method consists in using the cross- 
correlation function (Equation |4j in spherical harmonic 
space) to measure the presence of the ISW signal, how- 



One of the subtleties of the cross-correlation function 
method is the evaluation of the covariance matrix C covar 
and its inverse. This m atrix can be estim ated using the 
MCI or MC2 methods of iCabre et all (|2007|), in which case 



the test is strongly dependent on the quality of the simula- 
tions. Secondly, missing data will require extra care when 
estimating the power spectra, this can be tackled by us- 
ing MASTER (Monte Carlo Apodized Spherical Transform 
Estimato r) or QML (Quadratic Maximum Likelihood) 
meth ods ( |Hivon et al.|2002[|Efstathiou|2004]|Munshi et al.| 
20091. 

The spectra measurement can be used with one of four 
different statistical methods. The advantages and disadvan- 
tages of each method are summarised below and in Table [2j 
The first aims to detect a correlation between two signals, 
i.e. we test if the cross-power spectra is null or not. The 
second fits a (model-dependent) template to the measured 
cross-power spectra. The other two methods aim to validate 
a cosmological model as well as confirm the presence of a 
signal: the % 2 test and the model comparison. We describe 
them below: 

Simple correlation detection: The simplest and 
the most widely used method for detecting a cross- 
correlation between two fields X and Y (h ere suppos- 



ing that Y is correlated with X) (see e.g. Boughn fc 
Crittenden||2002l |Afshordi et al.||2004"l |Pietrobon et al. 
2006 " 

tion coefficient p(X,Y), defined as 



Sawang wit et al.||2010 | is to measure the correla- 



P(X,Y) 
where, Cor(A, Y) = 



Cor(A, F)/Cor(A, X) 



Ke[X*(p)Y(p)] , ( U ) 



with p a position or scale parameter and N p the number 
of considered positions or scales. There is a correlation 
between the two fields if p(X, Y) is not null. The corre- 
lation coefficient is linked to the cross-power spectra in 
harmonic space: 



space) to measure the presence of the ISW signa l, how- / \ / \ 

ever this approach has recently been challenged by |L6pez^| p(g, T) = [ Y^(2^ + l)C g T(£) ) / I ^(2^ + l)C gg (£) ] . 

Corredoira et al. (2010), because of its high sensitivity to \ i / \ £ I 

noise and fluctuations due to cosmic variance. (12) 



(i 
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Thus the nullity of the coefficient implies the nullity of 
the cross-power spectra. A z-score can be performed in 
order to test this nullity, 



(13) 



where the standard error of the correlation value, o~ p 
can be estimated using Monte Carlo simulations under 
a given cosmology. In the literature, most applications 
of this method assume that Kq follows a Gaussian dis- 
tribution u nder the null hypoth esis (i.e. no correlation), 



for example Vielva et al. ( 2006 ) and Giannantonio et al. 



(2008), which is not necessarily true. 
The distribution of the Kq test can be inferred if we as- 
sume that both fields X and Y, of the cross-correlation 
are Gaussian. In this case, the correlation coefficient dis- 
tribution is the normally distributed but follows a nor- 
mal product distribution, which is far from Gaussian. In 
the case where Y is a constant field, the correlation coef- 
ficient follows a normal distribution and the distribution 
of the hypothesis test Kq will depend on how the vari- 
ance of the estimator a p is computed. If this last value 
is derived from the observation X, then Kq follows a 
Student's t-distribution which converges to a Gaussian 
distribution only when p is high (by the central limit 
theorem). Otherwise, if the variance o~ p is estimated in- 
dependently from the data (through Monte-Carlo, for 
example) or known for a given cosmology, Kq can be 
assumed to follow a Gaussian distribution. This means 
that Kq is generally not Gaussian even if the correlation 
coefficient is Gaussian (see section Ej). 
This method does not include knowledge of the underly- 
ing ISW signal, nor of the galaxy field, though the error 
bars can be estimated from Monte Carlo simulations 
which include cosmological information. 
Amplitude estimation (or template matching): 
The principle of the amplitude estimation is to measure 
whether an observed signal corresponds to the signal 
predicted by a given cosmological model. 



The estimator and its variance are given by (e.g. Ho 
et al.||2008| |Giannantonio et al.|[2008| ): 



gT ^covar^gT 
^ gT ^covar'— ' r. 



/^Th*/-i-l oTh 
^ gT ^covar^pT 1 



(14) 

where Cj^ is the theoretical cross-power spectrum, 
C^ s the estimated (observed) power spectrum and 
Ccovar the covariance matrix calculated by Equation [9] 
A z-score, 

Ki = A>a , (15) 

is usually applied to test if the amplitude is null or not. 
Goodness of fit, \ 2 test: 



The goodness of fit or x 2 test is given by (e.g. |Afshordi 
elalp004{ [Rassat et al.|[2007l ): 



Ki 



V^gT ~ ^gT ) '-'covar K^gT ~ ^gT ) : 



where K 2 follows a x 2 distribution with number of de- 
grees of freedom (d.o.f) depending on the input data. 
This tests the correspondance of the data with a given 
cosmological model, but does not infer if the tested 
model is in fact the best. Equation 16 also assumes 



gives an idea on the probability of rejecting the model, 
but cannot be compared directly with the Ki value for 
the null hypothesis without careful statistics (see next 
method on model comparison). 
Model comparison: 

The model comparison method is based on the gener- 
alised likelihood ratio test and asks the question: 'Do the 
data prefer a given fiducial cosmological model over the 
null hypothesis ?'. This question is important because 
it could be possible to use the previous \ 2 test to detect 
an ISW signal - yet the data could still also be compat- 
ible with a null hypothesis (see for e.g. , lAfshordi et al 



20041 iRassat et al. 1120071 iFrancis & Peacock||2010b|). In 



this case it is important to perform a model compari- 
son to find which model is preferred by the data. Two 
hypotheses are built: 

— Hq\ "there is no ISW signal", i.e. the cross-power 
spectra is null; 

— H±: "there is an ISW signal compatible with a fidu- 
cial cosmology", i.e. the cross-power spectra is close 
to an expected one. 

One should then estimate: 



K 3 = A X 2 



(c° T bs m 



i 

covar 



\ gT 



1 

covar 



(C Th 



gT 



/-tObs\ 



(17) 



where K3 converges asymptotically to a \ 2 distribution. 
If the value is higher than a threshold (chosen for a re- 
quired confidence level), the Hq hypothesis is rejected. 
However, such method is difficult to use directly because 
of the small sample bias, K3 is not likely to follow a \ 2 
statistic. In the case of the ISW signal, the signal for 
a 'standard' fiducial cosmology (e.g., WMAP 7 cosmol- 
ogy) is so weak that it usually returns a lack of detection 
for current surveys - this may not be the case for future 
or tomographic surveys. Notice that this model compar- 
ison method can be seen as an improved version of the 
goodness of fit. 

3.2.4. Field to field comparison 

Instead of comparing the spectra, one can work directly 
with the temperature field to measure the presence of the 
ISW signal. The observable in this case is now the ISW 
temperature field (<5isw)j rather than the cross-correlation 
power spectra C 9 t(£). The observed CMB temperature 
anisotropies <5obs can be described as: 



jobs 



XS 



ISW 



- S 



thcr 



(18) 



that the C(£)'s are Gaussian variables. The value of x 



where <5isw is the ISW field and A its amplitude (normally 
near 1), St the primordial CMB temperature field, <5 ther 
represents fluctuations due to secondary anisotropies other 
than the ISW effect and Af represents noise. In the context 
of the ISW effect, which occurs only on large (linear) scales 
where noise and other secondary anisotropies are negligible, 
we have: 

c>obs — S T + A<5isw ■ (19) 

The main difference between the fields and spectra ap- 
proach is that the fields method requires an estimation of 
the ISW temperature field (few)- There are several meth- 
ods to calculate <5isw from a given matter overdensity map. 
The most accurate way to reconstruct the ISW signal is to 
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use information from the full 3-dimensional matter distri- 
bution, which in theory requires overlapping galaxy and 
weak lensing maps on large scales, in order to measure the 
galaxy bias. Thi s may be possible in the future with sur- 
veys like Euclid ( Refregier et al.|[2010 ). Assuming a simple 
bias relation, the matter field can also be estimate d di- 
rectly from galaxy surveys (see Granett et al. 2009 who 
did this for small patches on the sky), in the case where 
only the general redshift distribution of the galaxy survey 
is known, the ISW field 5\sw can be approximated directly 
from the galaxy and tempera t ure maps using (see 



et al. 1998 Cabre et al. 20071 Giannantonio et al 



Boughn 



2008): 



TSW 



c gg W 



9tn 



(20) 



where gi m are the spherical harmonic coefficients of the 
galaxy map, and a 1 ™ the coefficients of the ISW temper- 
ature anisotropy map. 

Another approach is to reconstruct the ISW map us- 
ing Equat ion Tj where is estima ted using the Poisson 
Equation ( Francis &: Peacock||2010a I . 

In general, it is assumed that on non-linear scales, the 
ISW is called the Rees-Sciama effect and will produce a neg- 
atively correlated signal d ue to non-linear growing modes 
of th e matter distribution ( Schaefer et al. |2010 Cai et al. 
2010). However, some non-linear modes could also be decay- 
ing for example due to major merge rs or tidal stripping, o r 
due to alternative cosmologies as in |Afshordi et aT^pOlT] ). 
In this case the signal could be positively correlated even 
on non-linear scales (from equation [I]) . The total ISW sig- 
nal (i.e., the signal which is positively correlated) would not 
necessarily be Gaussian in this case. 

In our approach, we do not model possible contributions 
from non-linear (growing or decaying) modes, but we allow 
quasi-linear modes in the data or simulations to produce a 
positively correlated ISW signal as an approximation; we 
do this since it is in practice very difficult to separate linear 
and quasi-linear modes. 

As for the spectra approach, there are several statistical 
methods available to qualify detection: 

Simple correlation detection: 

The simple correlation detection method described for 
the spectra comparison, can in fact also be considered 
as a field comparison. This is the only method which 
directly overlaps between both approaches. 
Amplitude estimation (or template matching): 
Using the Gaussian framework, given an ISW field, 
the amplitude A can be estimated with the corre- 



sponding maximum likelihood estimator (Hernandez- 
Monteagudo|2008| |Frommert et~aT|2008[ [Granett et aT 



2009): 



A 



^ISW^TT^OBS 



^ISW^TT^ISW 



(21) 



A signal is present if A is non null and a z-score, 

Ki = X/a % , (22) 

directly yields the confidence level in terms of a. 
Equation 21] implicitly assumes that the primordial 
CMB field 5t is a Gaussian random field (we discuss 
this further in section [4]). 



Goodness of fit, x 2 test: 

The x 2 good ness of fit with Hi (see Model compari- 
son in 



3.2.3) yields: 



K$ — (^isw — <5obs)*Ctt(^isw — <S bs) > (23) 

where K§ is a x 2 variable with number of d.o.f depend- 
ing on the input data. In this case, the test only returns 
the confidence of rejecting the null hypothesis Ho. As 
in the x 2 test for the spectra, precaution must be taken 
when comparing x 2 values for different models, by using 
an appropriate model comparison technique. We intro- 
duce this in section 31 



3.3. Pros and cons of each method 

We have identified two main classes of methods: either using 
power spectra or fields to measure the ISW signal. For each 
approach one can choose amongst several statistical tools 
to measure the significance of a correlation or validate si- 
multaneously a correlation and a model. The advantages 
and disadvantages of both approaches are summarised be- 
low and in the top part of Table [2] 

One of the main advantages of using the field approach 
is that it assumes only that the primordial CMB field 
comes from a Gaussian random process, which is largely 
believed to be true. In the other approach, the spectra are 
assumed to be Gaussian, which is not the case. Several stud- 



ies ( |Cole et al.|2001||Kayo et al.|2001||Wild et al.|2005[ ) have 
also shown that the matter overdensity exhibits a lognor- 
mal behavior on la rge scale. The bias introduced has been 
shown to b e small ( Bernardeau et al. 2002[ jHamimeche fc 
Lewis|20"08 ) , however this approach is still theoretically ill- 
motivated. The main advantage of the spectra method is 
the relative ease when calculating the spectra from incom- 
plete data sets, as tools are avai lable for calculating the 
spectra (see e.g. Efstathiou 2004). In the field approach, 
managing missing data is an ill-posed problem. 

We r emind that a pro blem is defined as a well-posed 
problem ( Hadamard|1902 1 if 1) a solution exists, 2) the so- 
lution is unique and 3) the solution depends continuously 
on the data (in some reasonable topology). Otherwise, the 
problem is defined as a ill-posed problem. With missing 
data, the second point cannot be verified. Reconstruction 
of the data also requires inversion of an operation (e.g., the 
mask), and is therefore an ill-posed inverse problem. Notice 
that most inverse problems are ill-posed (e.g. deconvolu- 
tion) . 



4. A rigorous method for detecting the ISW effect 

Having identified in the previous section the numerous 
methods used in the literature to detect and measure the 
ISW signal, as well as their relative advantages and disad- 
vantages, we propose here a complete and rigorous method 
for detecting and quantifying the signal significance. We 
describe this method in detail below and it is summarised 
in Figure [2j 

4.1. Motivation 

If we consider the pros and cons of each detection method 
shown in Tableland in Section 3.3 we remark first that 
any method based on the comparison of spectra makes 
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the demanding assumption that the C(£)'s be Gaussian, 
whereas methods based on field comparison require only 
the primordial CMB field to be Gaussian. Instead the fields 
method assumes only that the primordial CMB is Gaussian, 
so we recommend this method be used for an ISW analysis. 



J Empirical distribution 
- Normal distribution 
2a boundary (normal) 
2.5a boundary (empiric 




Fig. 1. Comparison of Gaussian PDF for estimator \/<j\ 
with its true estimated distribution. A 2a significance using 
the Gaussian PDF corresponds in fact to a 2.5a detection 
with the true distribution. (Calculation for 2MASS survey, 
see section [6]). 



Another key issue is the estimation of the signal sig- 
nificance. We find that most approaches in the literature 
assume that the probability distribution function (PDF) of 
the estimator is Gaussian. In Figure[l]we evaluate the esti- 
mator's PDF (under the null hypothesis) for the ISW signal 
due the 2MASS survey (see Section [6]) using a Monte-Carlo 
method (purple bars) and compare it with a Gaussian PDF 
(red solid line). The distributions' tails differ which leads 
to a bias in the confidence level for positive data (i.e. data 
with an ISW signal). For the 2MASS survey, a 2a detec- 
tion with a Gaussian assumption (vertical solid/green line) 
corresponds in fact to a 2.5a detection using the true un- 
derlying PDF. In this case, the signal amplitude is under- 
estimated with the Gaussian assumption. Such behavior 
has also been s tudied for marginal detections by |Bassett &: 
Afsh ordi1 ( |2010[ ). To avoid this bias, we recommend that the 
PDF be estimated, and not assumed Gaussian. 

Finally, there is not one 'ideal' statistical method. 
Different methods have both advantages and disadvantages 
and a combination of different methods can prove comple- 
mentary. 



4.2. The Saclay method 

We present here a new ISW detection method which uses 
the fields as input (and not the spectra). As we have seen, 
several statistical tests are interesting in the sense that they 
do not address the same questions. Therefore we believe 
that a solid ISW detection method should test: 



1. The correlation detection: this test is independent of the 
cosmology. 

2. The amplitude estimation: this will seek for a specific 
signal. 

3. The model comparison: this allows us to check whether 
the model with ISW is preferred to the model without 
ISW. 

Using the fields instead of the spectra means we must 
deal with the problem of missing data, which we solve with 
a sparse inpainting technique (see Appendix |A| . Such a 
method has alread y been applied with success for CMB 
lensing estimations ( jPerotto et al.|[20l0 ). 

The last important issue is how we estimate the final 
detection level. As explained before, the z-score asymptot- 
ically follows a Gaussian distribution and so a bootstrap 
or Monte-Carlo method is required to derive the correct 
p-value from the true test distribution. 



4.2.1. Methodology 

Our optimal strategy (summarised by Figure [2]) for ISW 
detection is the following: 

1. Apply sparse inpainting to both the galaxy and CMB 
maps which may have different masks, essentially recon- 
structing missing data around the Galactic plane and 
bulge. 

2. Test for simple correlation, using a double bootstrap 
(one to estimate the variance and another to estimate 
the confidence) and using the fields as input. This re- 
turns a model-independent detection level. 

3. Reconstruct the ISW signal using expected cosmology 
and the inpainted galaxy density map. 

4. Estimate the signal amplitude using the fields as input, 
and apply a bootstrap (or MC) to the estimator. This 
validates both the signal and the model. 

5. Apply the model comp arison test using the fields as in- 
put (see Section 4.2.2[ ), essentially testing whether the 
data prefers a fiducial (e.g., ACDM) model over the null 
hypothesis, and measure relative fit of models. 

As we have chosen to work with the fields, the very first 
step is to deal with the missing data. This is an ill-posed 
problem, which can be solved using sparse inpainting (see 
Appendix [A] for more details). This approach reconstructs 
the entirety of the field, including along the Galactic plane 
and bulge. We show in Section [5] that the use of sparse 
inpainting does not introduce a bias in the detection of the 
ISW effect. 

We then perform a correlation detection on the re- 
construct ed fi elds data using a double bootstrap (see 
Appendix B.l I. Experiments show that the bootstrap tends 



to over-estimate the confidence interval, especially when the 
p- values are small. This is why the obtained detection must 
be used as a indicator when near a significant value (for high 
p- values, the bootstrap remains accurate). 

The second test evaluates the signal amplitude, which 
validates both the presence of a signal and the chosen 
model. Bootstrapping test can also be used here as it has 
no assumption on the underlying cosmology. However, since 
the accuracy of bootstrap depends on the quantity of ob- 
served elements, it may become inaccurate for low p- value, 
i.e. when there is detection. In such case, Monte-Carlo (MC) 
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•Apply 

sparse 
inpainting 
to galaxy 
and CMB 
maps which 
may have 
different 
masks 



Inpaint Data 



Simple 
Correlatit 
Methoc 



• Use inpainted fields 

•Test for simple 
correlation 

•Apply double 
bootstrap to estimator 

•Search for signal 
independently of 
model 



J 



Estimate 
Amplitude 
Method 



• Use inpainted fields (not 
spectra) 

• Estimate the signal 
amplitude 

•Apply MC (or bootstrap) 
to estimator 

• Measure confidence 
level 

•Validate signal and 
model 




• Use inpainted fields 
(not spectra) 

•Compare model with 
fiducial cosmology 
with null hypothesis 

• Use MC for estimator 



V. 



• Measure relative fit 
models 



Fig. 2. Description of the steps involved in our method for detecting the ISW effect. Tests 1 to 3 are complementary and 
ask different statistical questions. 



will provide more accurate p-values and, for example, with 
10 6 MC simulations, we have an accuracy of about 1/1000. 

This second test compares the ISW signal with a fidu- 
cial model, but does not consider the possibility that the 
measured signal could in fact be consistent with the 'null 
hypothesis'. So even with a significant signal, a third test 
is necessary. This more pertinent question is addressed by 
using the 'Model Comparison' method (defined in Section 
4.2.2), for the first time using the fields approach. 



In conclusion, our method consists of a series of comple- 
mentary tests which together answer several questions. The 
first test seeks the presence of a correlation between two 
fields, without any referring cosmology. The second model- 
dependent test searches a given signal and tests its nullity. 
The third test asks whether the data prefers a fiducial ISW 
signal over the null hypothesis. 

4.2.2. 'Field' model comparison 

We define here the model comparison technique using the 
fields approach, which has until now not been used in the 
literature. Using a generalised likelihood ratio approach, 
the quantity to measure is: 



OBS°TT°OBS- 



ISW 



^obs)*Ctt(^fsw — (Sqbs) 



(24) 



Theoretically Kq convergences asymptotically to a x 2 vari- 
able with a certain number of rf.o./'s. As we only have one 
observation we cannot assume (asymptotic) convergence. 
We can however use a Monte Carlo approach in order to 
estimate the p-value of the test under the H hypothesis. 

The p-value is defined as the probability that under Hq 
the test value can be over a given Kg, i.e. P(t > K§) = 
J K p(x)dx, where p is the probability distribution of the 
test under the Hq hypothesis. By simulating primordial 
CMB for a fiducial cosmology, these values can be easily 
computed. Then the p-value gives us a confidence on re- 
jecting the H hypothesis. 



Notice that the same procedure for the p-value esti- 
mation ca n b e applied on K4 ( Equ ation |22[ ) , even on K\ 
(Equation 15) and K 3 (Equation ujh for the power spectra 
methods. We will further refer to this p-value estimation as 
the Monte-Carlo estimation, as we theoretically know the 
distribution under the null hypothesis, i.e. the primordial 
CMB is supposed to come from a Gaussian random process. 

5. Validation of the Saclay Method 

In order to validate the Saclay method, we estimate the 
detection level expected using WMAP 7 data for the CMB 
and 2MASS and Euclid data for the galaxy data (see sec- 
tion [6] for a description of WMAP and 2MASS data sets). 
We quantify the effect of the inpainting process on CMB 
maps with and without an ISW signal. We do this by sim- 
ulating 2MASS-like and Euclid-like Gaussian and lognor- 
mal galaxy distributions and WMAP7-like Gaussian CMB 
maps (using cosmological parameters from Table [6]) both 
with and without an ISW signal. We then apply our method 
to attempt a detection of the ISW signal. We do this both 
on full-sky maps as well as on masked data where we have 
reconstructed data behind the mask using the sparse in- 
pain tin g tec hnique (the masks we use are as described in 
6.1 and 



6.2) 



For each simulation, we run the 3-step Saclay method. 
Except for the cross-correlation method where we use 100 
iterations for the 2MASS-like and 1000 for Euclid-like sim- 
ulations for the p-value estimation and 201 for the vari- 
ance estimation (nested bootstrap), every other Monte- 
Carlo process was performed using 10 000 iterations All 
tests were performed inside the spherical harmonics domain 
with I e [2, 100] for 2MASS and t = [2,350] for a Euclid- 
like survey. 

For the Euclid-li ke survey, we consider a g alaxy distri- 
bution as defined in Amara & Refregier (2007), with mean 
redshift z m = 0.8 and slopes a = 2, ji = 1.5. We reconstruct 
the ISW effect created by the projected galaxy distribution 
of the Euclid survey, by considering only one large redshift 
bin. In the future, it could be possible to refine such a re- 



10 



F.-X. Dupe et al.: Measuring the Integrated Sachs Wolfe Effect 



constructed map by considering tomographic bins, or using 
information from the spectroscopic survey. As sky cover- 
age maps are not yet available for Euclid, we consider the 
same mask as for 2MASS and inpaint regions with miss- 
ing data following Section [63] We choose do to this, rather 
than simply assume a value for the fraction of sky covered 
(/sky), so as to consider more realistic problems relating to 
the shape of the mask, and to test our inpainting method. 

5.1. Expected level of detection: 

The expected detection levels (in units of a) are reported in 
Tabled (2MASS) and Table Ej (Euclid). Methods 1-3 corre- 
spond to the 3-step method described in Figure [2] where (b) 
and (MC) denote bootstrap and Monte Carlo evaluations 
of the variance and the p- value of the test. The p- values are 
converted as a a value using the following formula: 



s = V2 erf _1 (l -p) 



(25) 



where p is the p-value, s the corresponding cr-score and 
erf -1 the inverse error function. 

The 2MASS simulations (Table [3]) show that we expect 
the same level of significance for an ISW detection, whether 
the mass tracer follows a Gaussian or a lognormal distribu- 
tion. In either case the significance is low, around la ± la. 
This means that for 2MASS-like survey, we have a signal 
to noise ratio (S/N) around la. We see no major difference 
between the expected detection levels of M2 and M3. The 
only difference is for Ml (but there is still agreement with 
M2 and M3 within la error bars) - this may be due to 
the fact that the bootstrap technique is more efficient for 
Gaussian assumptions. 

We also apply our method to CMB simulations with no 
ISW signal present (2 left columns of Table [3]), and find 
a lower detection significance than when an ISW signal is 
present. This is true even when inpainting is used to recover 
missing data, showing that the inpainting method does not 
introduce spurious correlations. 

In any case, all methods suggest it is difficult to detect 
the ISW signal with high significance using the 2MASS data 
as a local tracer of the matter distribution. 



Gaussian density 


Method 


Monte-Carlo 


Bootstrap 


Cross-Correlation 
(Equation 11 ) 


0.46 ±0.28 


0.44 ±0.35 


Amplitude estimation 
(Equation 21 ) 


0.44 ± 0.30 


0.47 ±0.33 


Lognormal density 


Method 


Monte-Carlo 


Bootstrap 


Cross-Correlation 
(Equation 11 ) 


0.39 ±0.27 


0.38 ±0.31 


Amplitude estimation 
(Equation 21 ) 


0.41 ± 0.26 


0.40 ±0.30 



Table 5. Expected p- values for inpainted maps of ISW 
signal for a 2MASS-like local tracer of mass using Monte- 
Carlo or bootstrap methods for the first two steps of the 
Saclay method. 



In Table |4j we show that an Euclid-like surve y, which 
is optimally designed for an ISW detection (see, Douspis 



et al. 2008), permits a much higher detection than with a 
2MASS survey. As with 2MASS simulations, we notice that 
M2 and M3 return similar detection levels, which are lower 
than Ml. Inclusion of masked data reduced the significance, 
but our inpainting method does not introduce spurious cor- 
relations as inpainted maps with no ISW do not return a 
detection. For a Euclid-like survey with incomplete sky cov- 
erage, we can expect to show that the data prefers an ISW 
component over no dark energy (M3) at the 4.7<r level, and 
detect a cross-correlation signal at the ~ 7a level. We find 
no significant differences in the detection levels when the 
simulations are assumed lognormal or Gaussian. 

We also investigate the performance of the wild boot- 
strap method for the confidence estimation. Table [5] shows 
the p-values estimated using Monte-Carlo procedure and 
wild bootstrap for the first two methods of the 3-step Saclay 
method. Notice that the bootstrap results are almost equiv- 
alent to Monte-Carlo ones. We find the bootstrap method 
wasn't always reliable when the p-value becomes small, be- 
cause the precision of the bootstrap depends on both the 
number of bootstrap samples (as any MC-like process) and 
the number of observed elements. This last dependence 
makes the bootstrap uncertain when the detection is almost 
certain, that is why we consider the bootstrapped cross- 
correlation as an indicator that needs refinement when the 
results are very significant. 

5.2. Power of the tests 

In order to investigate the different strengths of each 
method, we also evaluate the rate of true positives vs. the 
number of false positives (i.e. false detections) - this infor- 
mation is summarised in Figure [3] which shows Receiver 
Operating Characteristic (ROC) curves. The construction 
of the ROC curve requires the computation of p-values for 
several simulated cases (i.e., simulations with and without 
an ISW signal), which are then sorted by value. For each 
p-value or threshold, the corresponding false positive and 
true positive rates are computed. 

Generally, a more sensitive method may be more per- 
missive and so will return a higher proportion of false de- 
tections. An ideal method will have a ROC curve above the 
diagonal from (0, 1) to (1, 1). Similarly, a poor detector will 
produce a curve below the diagonal, which corresponds to 
odds worse than tossing a coin. The X-axis corresponds to 
the false positive rate, i.e. the ratio of CMB maps without 
ISW where ISW signal is detected at the current threshold. 
The Y-axis corresponds to the true positive rate, i.e. the 
ratio of CMB maps with ISW where ISW signal is detected 
at the current threshold. We recall that a point on the ROC 
curve corresponds to a threshold. 

Figure [3] shows the ROC curves for three methods ap- 
plied to a 2MASS-like survey (Left) and a Euclid-like sur- 
vey (emphRight): fields model test (thin solid green), field 
amplitude estimation (dot-dashed blue), simple correlation 
(dashed red). For the 2MASS survey, all the methods are 
inside the la error bar of the field's amplitude estimation 
and so are nearly equivalent, i.e. no method performs better 
than the others. 

For the Euclid-like survey, the statistics return much 
better values then for 2MASS (i.e., the ROC curves are far 
from the diagonal). The simple correlation method will re- 
turn more false positives than the other two methods, which 
are nearly identical. The ROC curves for each method differ 
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2MASS / Method 


Full sky 


Inpainted 


Full sky 


Inpainted 






maps (with ISW) 


maps (with ISW) 


map (no ISW) 


map (no ISW) 


Gaussian matter density field 


1 


Simple correlation (b) 


1.12 ±0.98 


1.08 ±0.99 


0.78 ±0.81 


0.81 ±0.86 


2 


Field amplitude (MC) 


0.94 ±0.67 


0.91 ±0.65 


0.68 ±0.57 


0.71 ±0.58 


3 


Field model test (MC) 


0.94 ±0.67 


0.91 ±0.64 


0.69 ±0.57 


0.72 ±0.58 


Lognormal matter density field 


1 


Simple correlation (b) 


1.37 ± 1.51 


1.30 ± 1.37 


0.85 ± 1.05 


0.98 ± 1.34 


2 


Field amplitude (MC) 


0.97 ±0.62 


1.00 ±0.65 


0.69 ±0.54 


0.78 ±0.61 


3 


Field model test (MC ) 


0.97 ±0.62 


1.00 ±0.65 


0.69 ±0.53 


0.78 ±0.60 



Table 3. Expected detection level (units of a) of ISW signal for a 2MASS-like local tracer of mass assuming Gaussian 
and lognormal distribution for a fiducial cosmology (see Table [6]). Methods 1 — 3 represent the 3-step method. Angular 
scales included in the analysis are I = [2 — 100]. 





Euclid / Method 


Full sky 


Inpainted 


Full sky 


Inpainted 






maps (with ISW) 


maps (with ISW) 


map (no ISW) 


map (no ISW) 


Gaussian matter density field 


1 


Simple correlation (b) 


> 7 


6.98 ±2.30 


0.87 ±0.67 


0.99 ±0.72 


2 


Field amplitude (MC) 


> 5 


4.70 ± 2.40 


0.78 ±0.49 


0.88 ±0.54 


3 


Field model test (MC) 


> 5 


4.77 ±2.42 


0.78 ±0.48 


0.88 ±0.54 


Lognormal matter density field 


1 


Simple correlation (b) 


> 7 


6.29 ±2.60 


0.84 ±0.70 


0.77 ±0.72 


2 


Field amplitude (MC) 


> 5 


4.43 ±2.37 


0.75 ±0.54 


0.74 ± 0.54 


3 


Field model test (MC ) 


> 5 


4.43 ±2.37 


0.74 ±0.53 


0.74 ± 0.54 



Table 4. Expected detection level (units of a) of ISW signal for a Euclid-like local tracer of mass assuming Gaussian 
distribution for a fiducial cosmology (see Table p|. Methods 1 — 3 represent the 3-step method. Angular scales included 
in the analysis are £ = [2 — 300] . 



by more than la at some points and so different methods 
will perform differently. 



6. The ISW signal in WMAP7 due to 2MASS 
galaxies 

We apply the new d etection method des cribed in Section 
[| to WMAP7 data ( |Jarosik et aTJ|20To| and the 2MASS 
galaxy survey which has been extensively used as a tracer of 
mass for the ISW signal (see Table [TJ. We describe first the 
data in Sections 16. II and 1 6.21 In Section [6~3l we describe the 
inpainting process that we apply to both CMB and galaxy 
data. In Section |6~4} we present the detection results. 





0.0449 




0.266 


Ov 


0.734 


n 


0.963 


erg 


0.801 


h 


0.710 


T 


0.088 


w 


-1.00 




0.0 



Table 6. Best fit WMAP 7 cosmological parameters used 
throughout this paper. 



Method 


Inpainted 
maps (with ISW) 


Simple cross-correlation (Eq. Il3| 


1.30a ± 1.37 


Amplitude estimation (Eq. 2z| 


1.00a ±0.65 


Model selection (Eq. 23 \ 


1.00a ±0.65 



Table 7. Expected detections for 2MASS like survey (from 
lognormal results in Table [3]) . 



6.1. WMAP 

For the cosmic microwave background data, we use several 
maps from NASA Wilkinson Microwave Anisotropy Probe: 
the internal linear combination m ap (ILC) fo r years 5 and 
7 (W MAP5, |Komatsu et al.|[2009t (WMAP7, | Jaros i k et al. 
2010[ ) and the ILC map by |Delabrouille et al.||2009| ) which" 
was reconstructed using a needlets technique. We avoid re- 
gions which are contaminated by Galactic emission by ap- 
plying the Kq85 temperature mask - which roughly corre- 
sponds to the Kp2 mask from the third year release (see 
Figure[4]). We also substract the kinetic Doppler quadrupole 
contribution from the data. WMAP simulations used to 
produce Tables [3] and [4] use WMAP 7 best fit parameters 
for a flat ACDM universe (see Table [6}. 



6.2. 2MASS Galaxy Survey 

The 2 Micron All-Sky Survey (2MASS) is a publicly avail- 
able full-sky extended source ca talogue (XSC) selected in 
the near- IR ( |Jarrett et aT][2000| . The near-IR selection 
means galaxies are surveyed deep into the Galactic plane, 
meaning 2MASS has a very large sky coverage, ideal for 
detecting the ISW signal. 



Following (Afshordi et al. 20041, we create a mask 
to exclude regions of sky where XSC is unreliab le using 
the IR reddening maps of Schlegel et al. ( 1998 ). Using 
A k = 0.367 x E{B - V), Afshordi et al. (2004pmd a limit 
A K < 0.05 for which 2MASS is seen to 98% complete for 
K 2 o < 13.85, where K 2 q is the if^-band isophotal magni- 
tude. Masking areas with Ak > 0.05 leaves 69% of the sky 
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False positive rate False positive rate 

Fig. 3. ROC curves for the 3-step Saclay method for the 2MASS survey (Left) and the Euclid survey (Right). Note the 
axes in the right-hand panel are different than in the left-hand panel. The statistical methods correspond to: fields model 
test (thin solid green, Method 3 in Table J3l), field amplitude estimation (dot-dashed blue, Method 2), simple correlation 
(dashed red, Method 1). For the 2MASSsurvey, all the methods are inside the la error bar of the field's amplitude 
estimation and so are nearly equivalent. For the Euclid- like survey, the statistics return much better values than for 
2MASS (i.e., the ROC curves are far from the diagonal). The simple correlation method will return more false positives 
than the other two methods, which are nearly identical. The ROC curves of the simple correlation test differs sometimes 
by more than la at some points from the other two methods and so is expected to perform differently. 



and approximately 828 000 galaxies for the analysis 
Figure [4]) 



sec 



We use t he redshift distrib ution computed by 



et al. (20041 (and also used in Rassat et al. 2007), and in 
order to maximise the signal, we consider one overall bin 



for magnitudes 12 < K < 14. The red shift distribut i on for 

: 1 of |Rassat et al.| p007| 

(solid black line) and peaks at z 



0.073. The authorsalso 
showed that the small angle approximation could be used 
for calculations relating to 2MASS, so Equations 10 and [4] 
can be replaced by their simpler small angle form: 



et al.|[2002| |Gorski et al.||2005 ) with resolution correspond- 
ing to N5rDE=512. The top two figures show 2MASS data 
Afshordi| (left) and ILC map (right) with the masks in grey. The two 
figures in the middle show the reconstructed density fields 
for 2MASS data (left) and the ILC map (right). The bottom 
two figures show the reconstruction of the ISW field using 
the inpainted 2MASS density map and Equation 



20 (for 



C 9T{Z) = ^ 



/ drOD 2 H[f - 1]P 



£+1/2 



and 



C gg (£) = dr-^P 



e 2 



€+1/2 



(26) 
(27) 



We estimate the bias from the 2MASS galaxy power spec- 
trum using the cosmology in Table [6] and find b = 1.27 ± 
0.03, which is lower than that found inlRassat et al. (2007). 



6.3. Applying Sparse Inpainting to CMB and Galaxy data 

As discussed in Section [4j regions of missing data in galaxy 
and CMB maps constitute an ill-posed problem when using 
a 'fi eld' based method. We propose to use sparse inpainting 
(see Starck et al.|2010 and appendix [A} to reconstruct the 
field in the regions of missing data. 

We apply this method to both the WMAP7 and 2MASS 
maps, and the reconstructed maps are shown in Fig ure |4| 
All maps are pixelised using the HEALPix software ( Gorski 



clarity, the first two multipoles (I = 0,1) are not present in 
this map). 



6.4. ISW Detection using 2MASS and WMAP7 Data 



f We use the Saclay method (section |4 

\6 



on WMAP7 and 

2MASS data with 10 e iterations for the Monte-Carlo it- 
erations, 1000 iterations for both bootstrap and nested 
bootstrap of the correlation detection, and search for the 
ISW signal. In order to test the effect of including smaller 
(and possibly non-linear) scales, we perform the analysis for 
three different Granges: £ € [£ min , 50], [£ min , 100], [£ min , 200], 
where £ m i D = 2 or 3. We notice that inclusion or not of the 
quadrupole (£ — 2) can affect the significance level slighty, 
which is why we chose two values for £ m i n . We report our 
measured detection levels in Table[8j We recall the expected 
detections for 2MASS in Table [7J which can also be found 
in the more complete set of results presented in Table [3] 

The results in Table [8] are co mpatible with the r esults 
in Table [3] within the errors bars. Rassat et al. ( 2007[ ) used 
a spectra model-comparison method over £ = [3 • — 30] , and 
found that the model with dark energy was marginally pre- 
ferred over the null hypothesis. Over a range I — [2/3 — 50], 
using a fields model-comparison method, we find that a 
model with dark energy is preferred over the null hypothesis 
at the 1.1 — 1.8er level, depending on the map. More gener- 
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2 MASS ILC WMAP 7yr 




-1.7e-06 ^^^^^^^H ^^^^m 1 5e-()6 



Fig. 4. Top: 2MASS map with mask (left) and WMAP 7 ILC map with mask (right). Middle: Reconstructed 2MASS 
(left) and WMAP 7 ILC (right) maps using our inpainting method. Bottom: reconstructed ISW temperature field due 
to 2MASS galaxies, calculated using Equation [20] For better visualisation of the maps as an input of the Saclay method, 
we consider only the information inside £ G [2, 200]. 
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ally, our results are compatible with the earliest ISW mea- 



surement u sing 2MASS data ( |Afshordi et al.||2004} [Rassat 
et al. 2007) and lie in the 1.1 — 2. Oct range depending on 



the data and statistical test used. 

The simple correlation test tends to report marginally 
higher detection levels than the field amplitude and model 
comparison tests, and the model comparison test similar 
values as the field amplitude test, which is compatible with 
the predictions from the ROC curves in Figure [3j 

7. Discussion 

In this paper we have extensively reviewed the numerous 
methods in the literature which are used to detect and mea- 
sure the presence of an ISW signal using maps for the CMB 
and local tracers of mass. We noticed that the variety of 
methods used can lead to different and conflicting conclu- 
sions. We also noted two broad classes of methods: one 
which uses the cross-correlation spectrum as the measure 
and the other which uses the reconstructed ISW tempera- 
ture field. 

We identified the advantages and disadvantages of all 
methods used in the literature and concluded that: 

1. Using the fields (instead of spectra) as input required 
only the primordial CMB to be Gaussian. This requires 
reconstruction of the ISW field, which is difficult with 
missing data. 

2. The ill-posed problem of missing data can be solved 
using sparse inpainting, a method which does not intro- 
duce spurious correlations between maps. 

3. Assuming the estimator was Gaussian led to under- 
estimation of the signal estimation. 

4. A series of statistical tests could provide complementary 
information. 

This led us to construct a new and complete method 
for detecting and measuring the ISW effect. The method is 
summarised as follows: 

1. Apply sparse inpainting to both the galaxy and CMB 
maps which may have different masks, essentially recon- 
structing missing data around the Galactic plane and 
bulge. 

2. Test for simple correlation, using a double bootstrap 
(one to estimate the variance and another to estimate 
the confidence) and using the fields as input. This re- 
turns a model-independent detection level. 

3. Reconstruct the ISW signal using expected cosmology 
and the inpainted galaxy density map; 

4. Estimate the signal amplitude using the fields as input, 
and apply a bootstrap (or MC) to the estimator. This 
validates both the signal and the model. 

5. Apply the fields model comparison test, essentially test- 
ing whether the data prefers a given model over the null 
hypothesis, and measure relative fit of models. 

The method we present in this paper makes only one as- 
sumption: that the primordial CMB temperature field be- 
haves like a Gaussian random field. The method is general 
in that it 'allows' the galaxy field to behave as a lognormal 
field, but does not automatically assume that the galaxy 
field is lognormal. 

We first applied our method to 2MASS and Euclid sim- 
ulations. We find that it is difficult to detect the ISW sig- 
nificantly using 2MASS simulations, and find no difference 



between assuming the underlying galaxy field is Gaussian 
or lognormal, and only mild differences depending on the 
statistical test used. With a Euclid-like survey, we expect 
high detection levels, even with incomplete sky coverage - 
we expect ~ 7ct detection level using the simple correla- 
tion method, and ~ 4.7c detection level using the fields 
amplitude or method comparison techniques. These detec- 
tions levels are the same whether the Euclid galaxy field 
follows a Gaussian or lognormal distribution. Our results 
also show that the inpainting method does not introduce 
spurious correlations between maps. 

We applied this method to WMAP7 and 2MASS data, 
and found that our results were comparable with early de- 
tections of t he ISW signal usin g 2MASS data flAfshordi 



et al. 2004 Rassat et al. 20071 and lied roughly in the 



1.1 — 2. Oct range. These results are also compatible with 
the simulations we ran for the 2MASS survey. 

The last test we performed, the model comparison test, 
asks the much more pertinent question of whether the data 
prefers a ACDM model to the null hypothesis (i.e. no cur- 
vature and no dark energy). Using this test, we find a 
l.l — 1.8cr detection for ranges £ = [2/3 — 50] and 1.2— 1.9ct 
for ranges £ = [2/3 — 100/200], which is sometimes higher 
than what was previously reported in Rassat et al. (2007) 
using a spectra models comparison test, without sparse in- 
painting or bootstrapping. A by-product of this measure- 
ment is the reconstruction of the temperature ISW field due 
to 2MASS galaxies, reconstructed with full sky coverage. 

By applying our method to different estimations of 
CMB maps, we highlight the importance of component sep- 
aration on the ISW detection. Table[8]shows scores between 
1.1 — 2.0cr on different WMAP maps. We were also able to 
detect the ISW signal at 2.7cr using a another map (not 
presented in this paper). The influence of the component 
separation method on the quality of the estimation needs 
to be more deeply understood in future works. 
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Appendix A: Sparse Inpainting 

The goal of inpainting is to restore missing or damaged re- 
gions of an image, in such a way that the restored map 
has the same statistical prop erties as the underlying un- 
masked map ( |Elad et al.|2005 1 . Sparse Inpain ting has been 
proposed for filling the gaps in CMB maps ( |Abrial et al. 
|2007 2008[) and for we ak l ensing mass map re construction 
QPires et al.|2009||2010| ). In |Perotto et al.||2010[ ), it has been 
shown that the sparse inpainting method does not destroy 
the CMB weak lensing signal, and is therefore an elegant 
way to handle the mask problem. 

The inpainting problem can be defined as follows. Let 
X be the ideal complete image, Y the observed incomplete 



1 


http : / /www . icosmo . org 






2 


http : / / j starck . free . f r/isap . html 




3 


http : / /www . ipac . caltech . 


adu/2mass/ 
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Galaxy Survey 


2MASS 


CMB data 


WMAP ILC 7yr | WMAP ILC 5yr | Needlets ILC 5yr 


Scales interval 


I G [2, 50] 


Simple correlation detection 


1.3a 


1.3a 


1.2a 


Field amplitude 


1.3a 


1.3a 


1.1a 


Model comparison 


1.3a 


1.3a 


1.1a 


Scales interval 


£ e [2, too] 


Simple correlation detection 


1.5(7 


1.4a 


1.3a 


Field amplitude 


1.4a 


1.3a 


1.2a 


Model comparison 


1.4a 


1.3a 


1.2a 


Scales interval 


£ G [2, 200] 


Simple correlation detection 


1.4a 


1.4a 


1.3a 


Field amplitude 


1.4a 


1.3a 


1.2a 


Model comparison 


1.4a 


1.3a 


1.2a 


Scales interval 


£ G [3, 50] 


Simple correlation detection 


2.0a 


1.9a 


1.5a 


Field amplitude 


1.8a 


1.7a 


1.4a 


Model comparison 


1.8a 


1.7a 


1.4a 


Scales interval 


£ G [3, 100] 


Simple correlation detection 


2.0a 


2.0a 


1.5a 


Field amplitude 


1.9a 


1.7a 


1.5a 


Model comparison 


1.9a 


1.7a 


1.5a 


Scales interval 


£ G [3, 200] 


Simple correlation detection 


1.9a 


1.9a 


1.6a 


Field amplitude 


1.8a 


1.7a 


1.5a 


Model comparison 


1.8a 


1.7a 


1.5a 



Table 8. Detection levels obtained with different CMB maps and different scales intervals (considering with and without 
quadrupole) using the Saclay method described in [4] 



image (images can be fields on the sphere) and L the binary 
mask (i.e. L[k, I] = 1 if we have information at pixel (fc, /), 
L[k, I] = otherwise). In short, we have: Y = LX. 

Inpainting consists in recovering X knowing Y and L. 
The masking effect can be thought of as a loss of sparsity 
in the spherical harmonic domain since the information re- 
quired to define the map has been spread across the spher- 
ical harmonic basis. 

Sparsity means that most of the information is concen- 
trated in a few coefficients, which when sorted from the 
largest to the smallest, follow an exponent ial decay. More 
details can be found in ( Starck et al.||2010 ). In this paper, 
the chosen 'dictionary' is the spherical harmonic domain. 

Denoting the spherical harmonic basis as $ (so $ T is 
the spherical harmonic transform, i.e. the projector onto 
the spherical harmonic space), ||z||o the pseudo-norm, 
i.e. the number of non-zero entries in z and ||z|| the classical 
l 2 norm (i.e. \\z\\ 2 = X)fc( z fc)2)> we thus want to minimise: 



min||$ J X|| 



subject to || Y-LX \\e 2 < a, (A.l) 



where a stands for the noise standard deviation in the noisy 
case. It has also been shown that if X is sparse enough, the 
lo pseudo-norm can al so be replaced by the c onvex l\ norm 
(i.e. 1 1*| 1 1 = Y,k N) ( ponoho fc HuopOOl] ) . The solution 
of such an optimisation task can be obtained through an 
iterative thresholding algorithm called M CA (Elad et al. 
2005] |Fadili et al.|[2009| |Starck et al^[20To| : 



1. decomposing the signal Z on the dictionary $ to derive 
the coefficients a = <& T Z (i.e. a is the vector containing 
the ae m coefficients). 

2. threshold the coefficients: a = p(a, A), where the thresh- 
olding operator p can either be a hard thresholding (i.e. 
p(a 2 ,A) = oti if \oti\ > A and otherwise) or a soft 
thresholding (i.e. 

p(ai, A) = sign(ai)max(0, \ai\ — A)). The hard thresh- 
olding corresponds to the Iq optimisation problem while 
the soft-threshold solves that for l\. 

3. reconstruct Z from the thresholds coefficients a. 

The threshold parameter A„ decreases with the iteration 
number and it plays a role similar to the cooling parame- 
ter of the simulated annealing techniques, i.e. it allows the 
solution to escape from local minima. 

It has been shown in Abrial et al. ( |2008 1 that this in- 
painting technique leads to accurate CMB recovery results. 
More det ails relative to this optimis a tion problem can be 



found in Combettes & Wajs 



and theoretical justifica tions for CMB sparse inpainting in 



(2005); Starck et al. (2010) 



Rauhut & Ward (2010). The software is available in the 
Multi-Resolution on the Sphere (MRS) package]^] 



Appendix B: Bootstrapping detection tests 

Most detection methods rely on a z-score whose confidence 
is c omp uted assuming a Gaussian distribution (see sec- 
tion 



r 4 



(X n + Y - LX n ) 



3.2 1. This assumption is valid within the central limit 



(A. 2) theorem and so is not always true. Using an estimator of 



where the nonlinear operator A$ ,\{Z) consists in: 



5 http://jstarck.free.fr/mrs.html 
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the true distribution will give more reliable confidence lev- 
els. To estimate the true distribution, two approaches are 
generally used: Monte-Carlo simulations when a model and 
its parameters are available or non-parametric bootstraps 
methods. 

In order to build a fully non-parametric and cosmology 
independent correlation detection test, we propose to use 
the wild bootstrap in order to estimate both estimator vari- 
ance and confidence interval. The remaining question will 
be the reliability of such test when bootstraps are some- 
times known to underestimate confidence intervals. The 
answer to the question depends mainly on the choice of 
domain (e.g. pixels, spherical harmonics) and the method 
chosen for estimating. 

Working with correlated data is difficult and estimated 
values are generally different to the true ones. We chose 
to work in the spherical harmonic domain, which produces 
heteroskedastic (i.e., the power spectrum is scale depen- 
dent) and uncorrelated data. For heteroskedastic and un- 
corrected data, bootstraps methods like wild bootstrap 



can be used. Flachaire (2005); Davidson & Flachaire (2008) 
showed the good behaviour of wild bootstrap in many situa- 
tions. We propose here a short introduction to this method, 
which we use to estimate p- values and estimator variances. 



where * indicate the bootstrapped version of the sample 
and e is a random variable following a given distribution. 
The performance of the wild bootstrap mainly depends on 
the distribution of e, which must verify some conditions like 
having a null mean and a unit variance. 

For example Liu (19881 proposed to use Rademacher 
variables for the distribution, 



1 with probability 0.5 , 
— 1 with probability 0.5 . 



(B.5) 



Davidson & Flachaire (2008) showed that exact inference 
(up to a theoretical accuracy) is possible if 5f m is indepen- 
dent from all the disturbance ue m and if the test distri- 
bution is symmetric about 0. Moreover, the rate of con- 
vergence of the error in the rejection probabilities (ERP) 
is at most n~ 3 / 2 with symmetric errors and n" 1 / 2 with 
asymmetric errors (n is the size of the observed sample). 
The ERP is the difference between the actual rejection fre- 
quency under the null hypothesis and the level of the test 
(e.g. about 0.045 for a 2a detection). In other words, this 
is the precision error on the inferred p-value. Notice that it 
mainly depends on the sample size. 



B.l. Wild Bootstrap and Regression Model 



Bootstrap methods were introduced by Efron (1979) as a 
generalisation of the jackknife and its concept has been ex- 
tended to many situations. For the regression case with 
hetero skedastic data, wild bootstrap was developed by |Liu| 
(1988), who established the ability of wild bootstrap to 



provide refinements for the linear regression model with 
heteroskedastic disturbances. Mammen] ( |1993[ ) showed that 
the wild bootstrap was asymptotically justified, in the sense 
that the asymptotic distribution of various statistics is the 
same as the asymptotic distribution of their wild bootstrap 
counterparts. 



If you look at the correlation estimator (Equation 11) 
and the amplitude estimation (Equation[2l|), they both sup- 
pose an underlying linear model: 



pSj. 



(B.l) 



where 5° is the observed field which contains the signal 
of interest (e.g. CMB), 5 C a tracer of the signal of inter- 
est (e.g. matter density, ISW reconstruction), p is either 
the correlation coefficient and the amplitude of the signal 
(depending of the estimator) and o is the noise which is 
dependent of the position. 

The originality of the wild bootstrap relies in the fact 
that its data generating process (DGP) creates new samples 
without any prior on the distribution of the noise. This 
process uses the residual, i.e. U£ m = 8f m — pSf m , where p 
is the parameter estimate. Then, new samples are created 
using the following formula, 



Hm 



\/)° I 2 



(B.2) 
(B.3) 

(B.4) 



B.2. Parametric bootstrap 

The wild bootstrap is very useful when little is known about 
the noise. However, sometimes the noise distribution is well 
known and its parameters can be estimated from the obser- 
vations. In this case, it is possible to use this distribution 
in order to generate new samples. For example, if we model 
the CMB primordial distribution by a multivariate normal 
law, the u\ m can be generated as, u\ m ~ Af(0, Cqmb) where 
Ccmb is the power spectra of the primordial CMB. Notice, 
that using the theoretical CMB power spectra leads to a 
Monte-Carlo methods which is then dependent on the cho- 
sen cosmology. 

B.3. Estimating the estimator variance 

When the variance of an estimator is unknown, a bootstrap 
can be used to estimate it with the following scheme: 

1. Estimate the parameter vector p from the observed 
data; 



2. Create B bootstrap sampl es, u sing Equation |B. 2 
other method (see section B.2) 



3. For each bootstrap sample, estimate the parameter vec- 
tor P *, i e [i,B], 

4. Compute the estimation of the variance of the estima- 
tor, 



Var(p) 



1 



B 



B 
i=l 



*\2 



1 



ul^Pt- (B.6) 



B ^ 



B.4. Estimating the p-value of a test 

The estimation of the confidence interval of a hypothesis 
test requires the generation of samples under the null hy- 
pothesis. Inside the wild boo tstra p DGP, that leads to set 
p = when using Equation B.2 for new samples (it also 
remain true with a parametric bootstrap) . Using this prop- 
erty, the scheme for computing the p-value is the following, 
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1. Estimate the parameter ve ctor a nd its variance (e.g. us- 
ing the previous scheme in B.3), p and Var(/5); 

2. Compute the z-score, 



= /5/ v / Var( / 5) ; 



(B.7) 



3. Create B boo tstra p samples whic h follow the null hy- 
pothesis using B.2 (or section B.2) and p — ; 

4. For each bootstrap sample, estimate the parameter, p*, 
and its variance Var(p*), i € [1,B]; 

5. For each bootstrap sample, compute the correspondant 
z-score, 



6. Compute the bootstrapped p-value, 



(B.S 



(B.9) 



where I denotes the indicator function, which is equal 
to 1 when its argument is true and otherwise. 



Equation |B.9| can be a little too rough and may 
lead to inaccurate results when the answer is just be- 
tween two bootstrap samples. One way to redu ce this ef- 
fect is by smooth ing the empirical distribution ( |Racine fc 
MacKinnon 2007), for example with a Gaussian kernel, 



* , 1^ (r-rt 

i=0 x 



(B.10) 



where (p is the Gaussian cumulative distribution function 
and h the bandwidth of t he kernel (chosen depending on 
the required precision, see |Racine fc MacKinnon|2007| ). 

B.5. Reliability of bootstrap 

One can be perplexed by the efficiency and the reliability 
of the bootstrap method. Many studies have proved that 
these methods outperform asymptotic tests (i.e. using the 
asymptotic distribution, see Davidson k, MacKinnon|2 006 ) 
and sometimes perform almost as well as exact tests ( (i.e. 
with the true distribution, we refer t he reader to Efron|1987 



DiCiccio & Efron 1996 



Hall 1995, for more information) 



i'he efficiency is mostly dependent on three key elements: 1) 
the Data Generating Process (DGP) under both hypotheses 
(often generating under the alternative hypothesis is non 
trivial), 2) the number of observations and 3) the num- 
ber of bootstrapped samples. The variance on p is given 
by p(l — 100) /B, where B is the number of bootstraps. 
While the number of bootstrapped sample can be easily 
tackled, the DGP on the chosen method and the number 
of observation depends on many parameters (e.g. configu- 
ration, instruments, acquisition time). Even with the most 
efficient DGP, the precision of the bootstrap methods is 
limited by the number of observations, often in 0(l/y/n), 
sometimes 0(l/n). In other words, bootstrap methods are 
mostly inaccurate when the observed value is at the tail of 
the involved distribution (i.e. and so rare event). For ex- 
ample, a p-value estimated using 100 observations will be 
±0.1 and be inefficient for characterizing high confidence, 
but can still be useful for rejecting. 
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